在現代監控和可觀測性領域，Grafana 已成為不可或缺的視覺化平台。隨著組織規模的擴大和監控需求的增加，有效管理 Grafana 的資料來源變得越來越具有挑戰性。本文將開始深入探討如何利用 Terraform 實現基礎設施即程式碼（IaC）的方法，設計一個強大的頂層結構來管理 Grafana 資料來源。
在深入技術細節之前，讓我們再次強調為什麼選擇 Terraform 進行 Grafana 資料來源的 IaC 管理：
我們的頂層結構設計基於兩個核心概念：設定模板（Configs）和資料源（Datasources）。
設定模板定義了可重複使用的資料來源設定。這種方法允許我們創建高度標準化的設定，可以跨多個資料來源實例共享。
設定範本（Configs）
configs 部分定義了可重複使用的資料來源設定範本：
configs = {
prometheus_default = { ... }
prometheus_custom = { ... }
loki_default = { ... }
tempo_default = { ... }
}
每個設定範本包含 json_data 和 secure_json_data，允許我們定義資料來源的具體設定和敏感資訊。
datasources 部分定義了實際的資料來源實例：
datasources = {
// Prometheus Datasource
prometheus = {
Prometheus-1 = {
uid = "prometheus"
url = "http://prometheus:9090"
is_default = true
config = local.configs.prometheus_default
}
Prometheus-2 = {
uid = "prometheus-2"
url = "http://prometheus-2:9090"
config = local.configs.prometheus_default
}
}
// Loki Datasource
loki = {
Loki-default = {
uid = "loki"
url = "http://loki:3100"
config = local.configs.loki_default
}
}
// Tempo Datasource
tempo = {
Tempo-default = {
uid = "tempo"
url = "http://tempo:3200"
config = local.configs.tempo_default
}
}
}
這個結構的設計採用了多層嵌套的 map 結構，具有以下特點和優勢：
config = local.configs.prometheus_default 這樣的引用，可以輕鬆地在多個資料源之間共享設定。這減少了重複代碼，提高了維護性。這種結構允許我們為每種類型的資料源定義多個設定，每個資源都可以引用同一個設定範本。
現在，我們已經瞭解了 Grafana 中不同類型的資料源及其設定，接下來我們將結合先前所提到的全域動態生成資源的概念，實作我們的 Grafana 資料源 IaC 解決方案。這種方法將允許我們靈活地管理多種類型的資料源，並為每個資料源提供自定義的設定選項。
接下來我們實際執行指令建立 grafana_data_source 資源：
locals {
datasources = {
// Prometheus Datasource
prometheus = {
Prometheus-1 = {
uid = "prometheus"
url = "http://prometheus:9090"
http_headers = {
"X-Scope-OrgID" = "1"
}
is_default = true
config = local.configs.prometheus_default
}
Prometheus-2 = {
uid = "prometheus-2"
url = "http://prometheus-2:9090"
http_headers = {
"Authorization" = "Bearer your.prometheus_token"
}
config = local.configs.prometheus_default
}
}
// Loki Datasource
loki = {
Loki-default = {
uid = "loki"
url = "http://loki:3100"
http_headers = {
"X-Custom-Header" = "LokiValue"
}
config = local.configs.loki_default
}
}
// Tempo Datasource
tempo = {
Tempo-default = {
uid = "tempo"
url = "<http://tempo:3200>"
config = local.configs.tempo_default
}
}
}
configs = {
prometheus_default = {
json_data = {
http_method = "POST"
sigv4_auth = false
sigv4_auth_type = ""
sigv4_region = ""
}
secure_json_data = {}
}
prometheus_custom = {
json_data = {
http_method = "GET"
sigv4_auth = true
sigv4_auth_type = "default"
sigv4_region = "us-west-2"
}
secure_json_data = {
access_key = "your-access-key"
secret_key = "your-secret-key"
}
}
loki_default = {
json_data = {
max_lines = 1000
derived_fields = [
{
datasourceUid = "tempo"
matcherRegex = "[tT]race_?[iI][dD]\\"?[:=]\\"?(\\\\w+)"
name = "TraceID"
url = "$${__value.raw}"
}
]
}
secure_json_data = {}
}
tempo_default = {
json_data = {
tracesToLogsV2 = {
customQuery = true
datasourceUid = "loki"
filterBySpanID = false
filterByTraceID = false
query = "|=\\"$${__trace.traceId}\\" | json"
}
}
secure_json_data = {}
}
}
}
resource "grafana_data_source" "datasources" {
for_each = merge([
for type, sources in local.datasources : {
for name, source in sources : "${type}/${name}" => merge(source, { name = name, type = type })
}
]...)
type = each.value.type
name = each.value.name
uid = each.value.uid
url = each.value.url
is_default = lookup(each.value, "is_default", false)
http_headers = lookup(each.value, "http_headers", null)
json_data_encoded = jsonencode(each.value.config.json_data)
secure_json_data_encoded = jsonencode(each.value.config.secure_json_data)
}
這個 Terraform 資源展示了幾個進階技巧。它利用 for_each 和 merge 函數動態生成多個資料源，採用嵌套 map 結構組織配置，提高了代碼的可讀性和可維護性。通過 lookup 函數處理可選屬性，以及 jsonencode 函數處理複雜的 JSON 配置，使得資源定義更加靈活。這種方法大大簡化了 Grafana 資料源的管理，使得大規模部署和維護變得更加高效。
接著就讓我們實際執行指令建立資源：
terraform init
terraform apply
---
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# grafana_data_source.datasources["loki/Loki-default"] will be created
+ resource "grafana_data_source" "datasources" {
+ access_mode = "proxy"
+ basic_auth_enabled = false
+ http_headers = (sensitive value)
+ id = (known after apply)
+ is_default = false
+ json_data_encoded = jsonencode(
// ...
)
+ name = "Loki-default"
+ secure_json_data_encoded = (sensitive value)
+ type = "loki"
+ uid = "loki"
+ url = "<http://loki:3100>"
}
# grafana_data_source.datasources["prometheus/Prometheus-1"] will be created
+ resource "grafana_data_source" "datasources" {
+ access_mode = "proxy"
+ basic_auth_enabled = false
+ http_headers = (sensitive value)
+ id = (known after apply)
+ is_default = true
+ json_data_encoded = jsonencode(
// ...
)
+ name = "Prometheus-1"
+ secure_json_data_encoded = (sensitive value)
+ type = "prometheus"
+ uid = "prometheus"
+ url = "<http://prometheus:9090>"
}
# grafana_data_source.datasources["prometheus/Prometheus-2"] will be created
+ resource "grafana_data_source" "datasources" {
+ access_mode = "proxy"
+ basic_auth_enabled = false
+ http_headers = (sensitive value)
+ id = (known after apply)
+ is_default = false
+ json_data_encoded = jsonencode(
// ...
)
+ name = "Prometheus-2"
+ secure_json_data_encoded = (sensitive value)
+ type = "prometheus"
+ uid = "prometheus-2"
+ url = "<http://prometheus-2:9090>"
}
# grafana_data_source.datasources["tempo/Tempo-default"] will be created
+ resource "grafana_data_source" "datasources" {
+ access_mode = "proxy"
+ basic_auth_enabled = false
+ id = (known after apply)
+ is_default = false
+ json_data_encoded = jsonencode(
// ...
)
+ name = "Tempo-default"
+ secure_json_data_encoded = (sensitive value)
+ type = "tempo"
+ uid = "tempo"
+ url = "<http://tempo:3200>"
}
Plan: 4 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
grafana_data_source.datasources["tempo/Tempo-default"]: Creating...
grafana_data_source.datasources["loki/Loki-default"]: Creating...
grafana_data_source.datasources["prometheus/Prometheus-1"]: Creating...
grafana_data_source.datasources["prometheus/Prometheus-2"]: Creating...
grafana_data_source.datasources["tempo/Tempo-default"]: Creation complete after 0s [id=1:tempo]
grafana_data_source.datasources["loki/Loki-default"]: Creation complete after 0s [id=1:loki]
grafana_data_source.datasources["prometheus/Prometheus-1"]: Creation complete after 0s [id=1:prometheus]
grafana_data_source.datasources["prometheus/Prometheus-2"]: Creation complete after 0s [id=1:prometheus-2]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
接著我們到 Grafana 中確認是否成功建立：
在這個快速變化的技術世界中，我們的 Grafana 資料源管理方法不僅僅是一種技術實現，更是一種思維模式的轉變。通過將基礎設施即代碼（IaC）的理念與 Grafana 的強大功能相結合，我們開闢了一條通向更彈性的監控系統管理之路，為CI/CD 流程注入了更多可能性。
更重要的是，隨著資料量的爆炸性增長和監控需求的日益複雜化，這種方法提供了一個可擴展的框架，能夠輕鬆適應新的資料源類型和不斷演變的監控策略。它不僅提高了我們對系統的掌控能力，還為數據驅動決策提供了堅實的基礎，使得從海量數據中提取有價值的洞察變得更加容易。
iThome鐵人賽
看影片追技術